Controllable array sorting device

ABSTRACT

A controllable array sorting device is disclosed, including: an array of diverting mechanisms; and a processor coupled to the array of diverting mechanisms, wherein the processor is configured to: process an instruction to perform a sorting action on a target item, wherein the instruction specifies a selected at least subset of the array of diverting mechanisms to use to perform the sorting action; and in response to the instruction, cause the selected at least subset of the array of diverting mechanisms to perform the sorting action on the target item.

CROSS REFERENCE TO OTHER APPLICATIONS

This application claims priority to U.S. Provisional Patent ApplicationNo. 63/087,054 entitled CONTROLLABLE AIR JET SORTING DEVICE filed Oct.2, 2020 which is incorporated herein by reference for all purposes.

BACKGROUND OF THE INVENTION

One type of conventional sorting device relies on hyperspectral signalsthat are received from examined items. Upon receipt of a hyperspectralsignal from an item, the conventional sorting device will determinewhether to act on sorting the item or determine not to based on thehyperspectral signal. However, the conventional sorting device's act ofsorting an item is not dynamically tailored to that particular item.

BRIEF DESCRIPTION OF THE DRAWINGS

Various embodiments of the invention are disclosed in the followingdetailed description and the accompanying drawings.

FIG. 1 shows an embodiment of a controllable array sorting device.

FIG. 2 is a diagram that shows a linear array of diverting mechanismsbelonging to a controllable array sorting device.

FIG. 3 is a diagram that shows a non-linear array of divertingmechanisms belonging to a controllable array sorting device.

FIG. 4 is a diagram showing examples of modular controllable arraysorting devices.

FIG. 5A is a diagram showing an example of a sorting control device.

FIG. 5B is a diagram showing an example of a controllable array sortingdevice.

FIG. 6A is a flow diagram showing an embodiment of a process for using acontrollable array sorting device.

FIG. 6B is a flow diagram showing an embodiment of a process forperforming a sorting action by a controllable array sorting device.

FIG. 7 is a flow diagram showing an example of a process for determiningwhether to suppress performing corresponding sorting actions on a set oftarget items.

FIG. 8 is a flow diagram showing an example of a process for determiningwhether to suppress performing a sorting action on a target item.

FIG. 9 is a diagram showing an example of a set of neighboringnon-target items relative to a target item on a conveyor device.

FIG. 10 is a flow diagram showing an example of a process for selectinga subset of diverting mechanisms of a controllable array sorting devicefor performing a sorting action on a target item.

FIG. 11 is a diagram showing an example relationship between the polygonof a target item and the determined firing width as well as the firingduration of a controllable array sorting device.

FIG. 12 is a diagram showing two example conveyor devices and acontrollable array sorting device that is located at the junctionbetween the two conveyor devices.

FIG. 13 is a diagram showing two example conveyor devices and twocontrollable array sorting devices that are located at the junctionbetween the two conveyor devices.

FIG. 14 is a diagram showing two example conveyor devices and twocontrollable array sorting devices that are located at the junctionbetween the two conveyor devices.

FIG. 15 is a diagram showing an example of a detachable controllablearray sorting device.

FIG. 16 is a diagram showing an embodiment of an efficient materialrecovery facility with a single sorting line.

DETAILED DESCRIPTION

The invention can be implemented in numerous ways, including as aprocess; an apparatus; a system; a composition of matter; a computerprogram product embodied on a computer readable storage medium; and/or aprocessor, such as a processor configured to execute instructions storedon and/or provided by a memory coupled to the processor. In thisspecification, these implementations, or any other form that theinvention may take, may be referred to as techniques. In general, theorder of the steps of disclosed processes may be altered within thescope of the invention. Unless stated otherwise, a component such as aprocessor or a memory described as being configured to perform a taskmay be implemented as a general component that is temporarily configuredto perform the task at a given time or a specific component that ismanufactured to perform the task. As used herein, the term ‘processor’refers to one or more devices, circuits, and/or processing coresconfigured to process data, such as computer program instructions.

A detailed description of one or more embodiments of the invention isprovided below along with accompanying figures that illustrate theprinciples of the invention. The invention is described in connectionwith such embodiments, but the invention is not limited to anyembodiment. The scope of the invention is limited only by the claims andthe invention encompasses numerous alternatives, modifications andequivalents. Numerous specific details are set forth in the followingdescription in order to provide a thorough understanding of theinvention. These details are provided for the purpose of example and theinvention may be practiced according to the claims without some or allof these specific details. For the purpose of clarity, technicalmaterial that is known in the technical fields related to the inventionhas not been described in detail so that the invention is notunnecessarily obscured.

Embodiments of a controllable array sorting device are described herein.Sensed data associated with a set of items on a conveyor device isobtained. For example, the sensed data may comprise one or more imagesof the set of items. A target item that is to be targeted by acontrollable array sorting device is determined based at least in parton the sensed data. The controllable array sorting device comprises anarray of diverting mechanisms. In some embodiments, the array ofdiverting mechanisms may comprise one or more air orifices from whichrespective pressurized airflows can be emitted. In some embodiments, thearray of diverting mechanisms may comprise one or more mechanicalpaddles that can be actuated to move. At least a subset of the array ofdiverting mechanisms is selected based at least in part on an attributeof the target item. An instruction (e.g., control signal) is sent to orgenerated within the controllable array sorting device to cause thecontrollable array sorting device to use the selected at least subset ofthe array of diverting mechanisms to perform a sorting action on thetarget item (e.g., while the target item is on the conveyor device orafter the target item falls off the conveyor device). In response toreceiving the instruction (e.g., from the sorting control device, alocal processor, or a cloud server), the controllable array sortingdevice is configured to perform the sorting action on the target itemusing the selected at least subset of the array of diverting mechanisms.Examples of items on which to perform a sorting action by a controllablearray sorting device as described in various embodiments describedherein include but are not limited to recycling, garbage, e-waste,demolition waste, packages, luggage, manufacturing assembly lines, andproduce. Specifically, the sorting techniques introduced here areapplicable to any sorting process where one or more objects must beidentified and then manipulated in the context of a sorting line. In anembodiment related to shipping or storing packages, as a package movesalong a conveyor, a visual or non-visual sensor provides sensed datathat a processor utilizes to recognize that the object needs to bemanipulated. Such recognition can be based upon a physical attribute ofthe object, including but not limited to shape, color, non-visiblespectral analysis, identifying tags, or labels. Once a package isrecognized in this embodiment, the processor generates an instructionthat triggers a diverting mechanism to move the package out of thesorting line. Embodiments of a diverting mechanism to manipulate thepackage include one or more of robotic grapples, robotic pusher orsuction devices, gate or door mechanisms, magnetic couplers, or airjets.

In various embodiments, the controllable array sorting device is locatedat the end of the conveyor device from which items being transported bythe conveyor device fall off from the conveyor device. As the items falloff the conveyor device and before they land on another surface (e.g.,another conveyor device), the controllable array sorting device may beinstructed to use the selected subset of its diverting mechanisms toperform a sorting action on the target items to remove the target itemsfrom the stream of items. In some embodiments, the controllable arraysorting device is located over or to the side of the conveyor devicethat is transporting items. As the items are being transported along theconveyor device, the controllable array sorting device may be instructedto use the selected subset of its diverting mechanisms to perform asorting action on the target items to remove the target items from thestream of items being transported along the conveyor device. In variousembodiments, performing a sorting action on a target item includesremoving the target item from a stream of materials by deflecting thetarget item into a collection container or onto a target conveyor usingat least a subset of the diverting mechanisms of the controllable arraysorting device. In various embodiments, “target” items can be eithermaterials that are desired to be recovered (e.g., recyclable materials)or materials that are not desired to be recovered (e.g., contaminantitems), depending on the application of the sorting device. As will bedescribed in further detail below, the at least subset of the divertingmechanisms of the controllable array sorting device is dynamicallyselected such that target items on which sorting actions are to beperformed can be successfully deflected into collection containers andwith a reduced likelihood that a nearby non-target item will also beinadvertently deflected into the collection containers.

FIG. 1 shows an embodiment of a controllable array sorting device. FIG.1 shows a controllable array sorting device, controllable array sortingdevice 100. In various embodiments, a “controllable array sortingdevice” comprises a device that includes a structure that supports oneor more arrays of diverting mechanisms that can be activated to deflecta target item into a collection container. In the example of FIG. 1,controllable array sorting device 100 comprises diverting mechanismarray 102 (and a local controller/processor that activates divertingmechanism array 102, which is not shown in FIG. 1). In the example ofFIG. 1, diverting mechanism array 102 includes 12 diverting mechanisms,where each is represented by a black dot. While controllable arraysorting device 100 includes one array of 12 diverting mechanisms, inactual practice, a controllable array sorting device may include anynumber of diverting mechanisms in any number of arrays. In someembodiments, at least some of the diverting mechanisms are air orificesthat are connected to one or more pressurized air source(s) (not shownin FIG. 1). The air orifices are controlled by a localcontroller/processor, which is configured to cause pressurized air to beemitted from the air orifices. For example, an “air orifice” comprisesan air opening or an air knife (e.g., a series of small air openings).In some embodiments, at least some of the diverting mechanisms arepaddles that are connected to a local controller/processor (not shown inFIG. 1), which is configured to activate the paddles by causing thepaddles to swing out and then retract. Other embodiments of a divertingmechanism to manipulate a target item include one or more of roboticgrapples, robotic pusher or suction devices, gate or door mechanisms,rotating friction wheels, magnetic couplers, or air jets. As shown inFIG. 1, controllable array sorting device 100 is configured to be placednear an end of conveyor device 104 and is configured to deflect targetitems (e.g., such as item 105) that fall off that end of conveyor device104 (e.g., by shooting air or by making physical contact). The fired-ontarget items are deposited into and/or transported into a collectioncontainer for storing recovered materials.

Controllable array sorting device 100 is configured to communicate withobject recognition device 106, which is located above conveyor device104 (or in any position providing sensor coverage of conveyor device104), and sorting control device (not shown). Object recognition device106 includes one or more sensors (e.g., an infrared camera, visualspectrum camera, volumetric sensor, or some combination thereof) thatpoint down towards conveyor device 104 and have a clear view of theitems being transported along conveyor device 104. In some embodiments,the items are sensed by a series of different sensors, where a latersensor in the series could refine data sensed by an earlier sensor inthe series. Object recognition device 106 is configured tocontinuously/periodically capture sensed data (e.g., one or more images)of the items that are transported by conveyor device 104 towardcontrollable array sorting device 100. In some embodiments, objectrecognition device 106 and/or the sorting control device are combinedwith controllable array sorting device 100. In some embodiments, objectrecognition device 106 is configured to send sensed data to a sortingcontrol device (not shown in FIG. 1) for the sorting control device toanalyze each set of sensed data using a trained machine learning modelto identify one or more target items among the items shown in the senseddata. The sorting control device is then configured to send tocontrollable array sorting device 100 control signals indicating thetarget items that controllable array sorting device 100 should perform asorting action on after the target items fall off of conveyor device104. In some embodiments, controllable array sorting device 100performing a sorting action on a target item is sometimes referred to as“firing on” the target item. In one specific example, the sortingcontrol device is configured to identify target items based on thedetermined material type associated with those target items. Forexample, target items are associated with material types that aredesirable (e.g., recyclable). In another specific example, the sortingcontrol device is configured to identify target items based on theshape, dimensions, and/or color of the items. For example, target itemsare a cluster of objects (e.g., a cluster of green beans) that match aset of desirable criteria. In some embodiments, the sorting controldevice is configured to determine one or more of the following pieces ofinformation: where a target item is located along the width of conveyordevice 104, when object recognition device 106 has obtained sensed dataassociated with the target item, and the approximate geometry (e.g.,length and width) and/or mass of the target item. Based at least on thedetermined information, the sorting control device is configured todetermine when a target item is to fall off of conveyor device 104 and,at that point, how controllable array sorting device 100 should use itsdiverting mechanisms to perform the sorting action on (e.g., to deflect)the target item. In various embodiments, a first aspect of controllinghow controllable array sorting device 100 should use its divertingmechanisms to perform a sorting action is determining how many divertingmechanisms (which is also sometimes referred to as the “firing width”)and which diverting mechanisms of its array will be used to deflect atarget item. In some embodiments, a second aspect of controlling howcontrollable array sorting device 100 should use its divertingmechanisms to perform a sorting action is determining for how long(which is also sometimes referred to as the “firing duration”) theselected diverting mechanisms will be activated to deflect the targetitem. For example, if the diverting mechanisms of controllable arraysorting device 100 were air orifices, then a greater number of divertingmechanisms is selected and/or caused to shoot air for a longer durationfor a target item that is determined to have a larger geometry (e.g., alarger bounding box) and/or mass than for a target item that isdetermined to have a smaller geometry (e.g., a smaller bounding box)and/or mass. In another example, controllable array sorting device 100is configured to activate diverting mechanisms located on its left sideif the target item is located on that side of conveyor device 104. Invarious embodiments, a third aspect of controlling how controllablearray sorting device 100 should use its diverting mechanisms to performa sorting action is determining when its selected diverting mechanismswill be activated to deflect a target item.

In some embodiments, additional factors that the sorting control devicemay use to control how controllable array sorting device 100 will useits diverting mechanisms (e.g., when to fire) include the (e.g.,detected) speed of conveyor device 104, heuristics (e.g., theorientation of the target item, the type of material of the targetitem), and the distance between controllable array sorting device 100and object recognition device 106.

In response to receiving a control signal from the sorting controldevice, controllable array sorting device 100 is configured to activatethe selected subset of diverting mechanisms that are instructed by thecontrol signal to perform a sorting action on a target item at a timeafter which the target item has fallen off of conveyor device 104. In afirst example, if the diverting mechanisms in controllable array sortingdevice 100 are air orifices, then the selected subset of air orificeswould be activated by releasing pressurized airflows out of those airorifices so that the airflows can hit the target item and deflect thetarget item into a collection container. In a second example, if thediverting mechanisms in controllable array sorting device 100 arepaddles, then the selected subset of paddles would be activated byactuating the paddles so that they swing towards the target item todeflect the target item into a collection container.

In some embodiments, controllable array sorting device 100 may not beinstructed by the sorting control device to perform a sorting action ona target item at all if suppression criteria with respect to the targetitem is met. As described above, the selected diverting mechanisms ofcontrollable array sorting device 100 are to physically deflect (e.g.,using a vacuum, a positive airflow, or a paddle making contact with thetarget item) the target item into a collection container. However, it ispossible that the physical deflection aimed at the target item couldinadvertently also deflect a non-target item into the collectioncontainer intended for the target item. The result of inadvertentlydeflecting a non-target item into the collection container is that thepurity level of items collected in one or more collection container(s)would decrease, which is undesirable. For example, the “purity level”corresponding to items deposited into one or more collection containerscan be defined as either 1) the total number of collected target itemsover the total number of all items collected at the collectioncontainer(s) or 2) the total weight of collected target items over thetotal weight of all items collected at the collection container(s).Typically, the greater the purity level of items collected at thecollection container(s), the greater the (e.g., economic) value that isassociated with the collected items. As such, it is undesirable to allowneighboring non-target items to be inadvertently deposited into acollection container when controllable array sorting device 100 performsa sorting action on a target item because doing so will lower the puritylevel associated with items collected at the collection container. Aswill be described in further detail below, in some embodiments, asorting control device is configured to determine a set of non-targetitems that are neighbors to a target item on a conveyor device based onthe sensed data. Then, the sorting control device is configured todetermine whether a controllable array sorting device should perform asorting action on the target item based on comparing attributeinformation associated with the target item with attribute informationassociated with the set of neighboring non-target items relative to thetarget item. If the sorting control device determines that thecontrollable array sorting device should perform a sorting action on thetarget item, then the sorting control device will instruct thecontrollable array sorting device to perform a sorting action withrespect to the target item. Otherwise, if the sorting control devicedetermines that the controllable array sorting device should not performa sorting action (thereby, “suppressing” fire) on the target item, thenthe sorting control device will not instruct the controllable arraysorting device to perform a sorting action with respect to the targetitem.

While the diverting mechanisms of controllable array sorting device 100shown in FIG. 1 point downwards toward conveyor device 104, in someembodiments, the diverting mechanisms of a controllable array sortingdevice can be configured to point in any other direction relative to theadjacent conveyor device. In some embodiments, each diverting mechanismof the array can be actuated to rotate around an axis in order todeflect target items into different directions.

In some embodiments, the height at which controllable array sortingdevice 100 is placed relative to the surface of conveyor device 104depends on the type of diverting mechanisms that is included indiverting mechanism array 102. For example, if the diverting mechanismsof diverting mechanism array 102 included air orifices that can emitpositive airflows, then controllable array sorting device 100 can be setto be higher relative to the surface of conveyor device 104 because theemitted airflows can deflect the trajectory of target items that falloff of conveyor device 104 without the air orifices needing tophysically contact the target items. In contrast, if the divertingmechanisms of diverting mechanism array 102 included actuatable paddles,then controllable array sorting device 100 would need to be set to belower relative to the surface of conveyor device 104 because when thepaddles are actuated to deflect the target items, the actuated paddlesneed to directly contact the target items.

While controllable array sorting device 100 shown in FIG. 1 features anarray of diverting mechanisms, in some other embodiments, a controllablearray sorting device may include more than one array of divertingmechanisms that form a two-dimensional plane of diverting mechanisms ormultiple planes of diverting mechanisms that form a three-dimensionalarrangement of diverting mechanisms. In some embodiments, a controllablearray sorting device may include more than one array of divertingmechanisms and where each array of diverting mechanisms points in adifferent direction (e.g., because different types of target items mayneed to be deflected in different directions toward the respectivelocations of their collection containers). In some embodiments,individual or groups of diverting mechanisms can be controlledindependently or in combination to move target objects through thethree-dimensional space. In such embodiments, the sorting control deviceis configured to send control signals to such subsets of the divertingmechanism array in order to effectuate the desired motion of the targetobject. In addition to spatial control of diverting mechanism array 102,the sorting control device is configured to provide control signals thatmay vary over time or array location based upon the control system logicimplemented. Examples of control system logic implemented to provideappropriate control signals are Kalman filters, traditional feedbackcontrol (e.g., PID controller), or alternative control algorithms basedupon machine learning.

In some embodiments, diverting mechanism array 102 of controllable arraysorting device 100 may include one or more linear (straight) rows ofdiverting mechanisms (as shown in FIG. 2, below). In some embodiments,diverting mechanism array 102 of controllable array sorting device 100may include one or more non-linear (e.g., a curved) rows of divertingmechanisms (as shown in FIG. 3, below).

FIG. 2 is a diagram that shows a linear array of diverting mechanismsbelonging to a controllable array sorting device. In some embodiments,diverting mechanism array 102 of controllable array sorting device 100of FIG. 1 may be implemented using one or more of array of divertingmechanisms 200. Array of diverting mechanisms 200 includes a linear rowof diverting mechanisms. In the example of FIG. 2, each divertingmechanism of array of diverting mechanisms 200 such as divertingmechanism 202 is an air orifice that is connected to a (pressurized) airsource. Given the linear arrangement of the diverting mechanism of arrayof diverting mechanisms 200, each diverting mechanism (air orifice) canemit an airflow at a target item to deflect the target item in adirection that is orthogonal to the plane that includes array ofdiverting mechanisms 200. For example, an air orifice comprises an airopening. In another example, the air orifice may comprise an air knife,which comprises a series/collection of small air apertures.

FIG. 3 is a diagram that shows a non-linear array of divertingmechanisms belonging to a controllable array sorting device. In someembodiments, diverting mechanism array 102 of controllable array sortingdevice 100 of FIG. 1 may be implemented using one or more of array ofdiverting mechanisms 300. Array of diverting mechanisms 300 includes anon-linear (curved) row of diverting mechanisms. In the example of FIG.3, each diverting mechanism of array of diverting mechanisms 300 such asdiverting mechanism 302 is an air orifice that is connected to a(pressurized) air source. Given the non-linear arrangement of thediverting mechanism of array of diverting mechanisms 300, one or morediverting mechanisms (air orifices) can emit (e.g., a combination of)airflows at a target item to deflect the target item in a particulardirection away from the plane that includes array of divertingmechanisms 300. For example, an air orifice comprises an air opening. Inanother example, an air orifice may comprise an air knife, whichcomprises a series/collection of small air apertures.

FIG. 4 is a diagram showing examples of modular controllable arraysorting devices. Manufacturing controllable array sorting devices with apredetermined number of diverting mechanisms can be easier accomplishedat scale, rather than manufacturing controllable array sorting deviceswith a variable/custom number of diverting mechanisms for each useparticular application of the controllable array sorting device. Assuch, in some embodiments, a controllable array sorting device can beimplemented in “modular form,” where each controllable array sortingdevice includes one of a predetermined number of diverting mechanisms.The predetermined number of diverting mechanisms can be selected to be aminimum unit or number of diverting mechanisms that is expected for anapplication. Then, for each use case, one or more modular controllablearray sorting devices can be employed (e.g., arranged side-by-side orfront-to-back at the end of a conveyor device) to perform sortingactions on target items that fall off of a conveyor device. Whenarranged side-by-side, the arrays of diverting mechanisms across all ofthe modular controllable array sorting devices can collectively form oneaggregated array of diverting mechanisms that spans at least the widthof the conveyor device. When arranged front-to-back, the arrays ofdiverting mechanisms across all of the modular controllable arraysorting devices can collectively form one two-dimensional plane ofdiverting mechanisms that comprises a width of at least two arrays ofdiverting mechanisms. In some embodiments, the combination of more thanone modular controllable array sorting device can be treated as a singlecontrollable array sorting device (e.g., the arrays of divertingmechanisms across multiple modular controllable array sorting devicescan be controlled by a single sorting control device in performing thesorting action on a target item). Besides cost savings in manufacturingmodular controllable array sorting devices, multiple modularcontrollable array sorting devices can also be advantageously arrangedside-by-side at different angles relative to each other so as to form anaggregated array of diverting mechanisms across the sorting devices thatis of a non-linear shape, if desired.

In the example of FIG. 4, modular controllable array sorting device 402a and modular controllable array sorting device 402 b each includes anarray of six diverting mechanisms. Modular controllable array sortingdevice 402 a and modular controllable array sorting device 402 b can bethought of as two instances of the same type of modular controllablearray sorting device. As shown in the example of FIG. 4, modularcontrollable array sorting device 402 a and modular controllable arraysorting device 402 b are placed side-by-side relative to each other at aslight angle so as to form a non-linear aggregated array of twelvediverting mechanisms across the two modular controllable array sortingdevices. Additional instances of the modular controllable array sortingdevice can be placed on either side of modular controllable arraysorting device 402 a and modular controllable array sorting device 402 bso as to increase the length of the aggregated array of divertingmechanisms that is collectively formed by all of the modularcontrollable array sorting devices.

FIG. 5A is a diagram showing an example of a sorting control device. Insome embodiments, a sorting control device portion of FIG. 1 may beimplemented using sorting control device 500 of FIG. 5A. In the exampleof FIG. 5A, sorting control device 500 includes neural processing logic502 and sorting logic 504. Each of neural processing logic 502 andsorting logic 504 may either be implemented together on a commonphysical non-transient memory device, or on separate physicalnon-transient memory devices. In various embodiments, each of neuralprocessing logic 502 and sorting logic 504 may be implemented using oneor more microprocessors coupled to one or more memories that areprogrammed to execute code to carry out the functions of correspondingneural processing logic 502 and sorting logic 504 described herein. Inother embodiments, each of neural processing logic 502 and sorting logic504 may additionally, or alternately, be implemented using anapplication specific integrated circuit (ASIC) or field programmablegate array (FPGA) that has been adapted for machine learning.

Neural processing logic 502 is configured to receive raw sensed data(which in the case of a camera sensor may comprise image frames, forexample) from an object recognition device. Neural processing logic isconfigured to provide the received sensed data as input to one or moreneural network and artificial intelligence techniques to locate andidentify items appearing within the image frames that are potentiallytarget items. As the term is used herein, an “image frame” is intendedto refer to a collection or collected set of sensed data captured by anobject recognition device that may be used to capture the spatialcontext of one or more potential items on a conveyor device along withcharacteristics about the items. A feed of image frames captured by anobject recognition device is fed, for example, to a machine learninginference technique. The sequence of captured image frames may beprocessed by multiple processing layers, or neurons, to evaluate thecorrelation of specific features with features of items that it haspreviously learned. Alternative techniques to detect items within animage include Fully Convolutional Neural Network, Multibox, Region-basedFully Convolutional Networks (R-FCN), Faster R-CNN, and other techniquessuch as object detection, instance-aware segmentation, or semanticsegmentation techniques.

Among the recognized items, neural processing logic 502 is configured todiscriminate between target and non-target items based on areconfigurable and dynamic set of target item criteria. For example, theset of target item criteria can describe which types of items areconsidered to be “target items” (e.g., and therefore should have sortingactions be performed on them), while items that do not match the set oftarget item criteria are considered to be “non-target items” (e.g., andtherefore should not have sorting actions be performed on them). Forexample, the set of target item criteria may describe a set of attributeinformation (e.g., target material type, and/or other target itemshape/dimensions/color) for which a matching item is considered to be a“target item.” For example, a matching item can be a single object(e.g., an aluminum can) or a cluster of objects (e.g., a cluster ofgrapes or a cluster of green beans). In various embodiments, the set oftarget item criteria that is configured for a particular sorting controldevice may be specifically configured for that particular sortingcontrol device and may differ from the set of target item criteria thatis configured for a different sorting control device. Put another way,different sorting control devices may consider different types of itemsto be target items depending on their respective sets of target itemcriteria. In various embodiments, a sorting control device's set oftarget item criteria may be reconfigured/updated/modified to cause thesorting control device to identify different items to be “target items”over time. In some embodiments, in response to detected events, tunableparameters associated with the material recovery facility, including asorting control device's set of target item criteria can bereconfigured.

Based on the input sensed data (e.g., image frames) that is provided byan item recognition device, neural processing logic 502 is configured todetermine information related to target items and non-target items thatare being transported by the conveyor device. In some embodiments, theinformation related to target items that are determined by neuralprocessing logic 502 includes attribute information. For example,attribute information includes one or more of, but not limited to, thefollowing: a material type associated with each item, an approximatemass associated with each item, an associated geometry associated witheach item, dimensions (e.g., height and width/area) associated with eachitem, a designated deposit (e.g., collection container) locationassociated with each item, and an orientation associated with each item.In some embodiments, the information related to items that aredetermined by neural processing logic 502 includes location information.For example, location information includes one or more coordinates atwhich each item was located in the image frame(s) that were input intoneural processing logic 502. In a specific example, the locationinformation of each item is the coordinate of the centroid of the item.Neural processing logic 502 is configured to send the attributeinformation and location information of target items and non-targetitems to sorting logic 504.

Sorting logic 504 is configured to select, for each target item that hasbeen identified by neural processing logic 502, at least a subset of aset of diverting mechanisms that is associated with the controllablearray sorting device to use to perform a sorting action on that targetitem. A particular subset of the diverting mechanisms is dynamicallyselected to perform a sorting action on each target item so as to ensurea sufficiently strong but focused deflection of the target item into acollection mechanism. In some embodiments, the selected subset ofdiverting mechanisms includes a number of contiguous divertingmechanisms as well as the location along the array of divertingmechanisms at which the selected number of diverting mechanisms arelocated. In some embodiments, the number of contiguous divertingmechanisms (which is sometimes called a “firing width”) that is selectedby sorting logic 504 is based at least in part on an attribute of thetarget item. For example, the number of contiguous diverting mechanismsthat is selected by sorting logic 504 is based at least in part on anapproximate mass or an associated geometry of the target item. Forexample, the larger the mass or the larger the associated geometry ofthe target item, the greater the number of diverting mechanisms thatwould be selected by sorting logic 504 to perform the sorting action. Insome embodiments, where along the array of diverting mechanisms of thecontrollable array sorting device the selected number of divertingmechanisms is located is determined based at least in part on thelocation information (e.g., centroid) of the target item on the surfaceof the conveyor device. For example, the location of the target itemalong the width of the conveyor device is used by sorting logic 504 todetermine the selected number of diverting mechanisms of thecontrollable array sorting device that are (e.g., have an averagelocation that is) the closest to the location of the target item. Insome embodiments, where the controllable array sorting device includesmore than one array of diverting mechanisms and where the divertingmechanisms in different arrays are configured to deflect target items indifferent directions, sorting logic 504 is also configured to select thelocation of the selected number of diverting mechanisms among aparticular array that is configured to deflect the target item into adirection in which a collection container corresponding to that targetitem is located.

In some embodiments, where the diverting mechanisms of the controllablearray sorting device can be activated to continuously or repeatedlyperform the sorting action for a duration of time, sorting logic 504 isconfigured to determine a duration of time during which the selectedsubset of diverting mechanisms is to be activated. For example, wherethe diverting mechanisms of the controllable array sorting devicecomprise air orifices that are connected to pressurized air sources, aduration of time during which the selected diverting mechanisms (airorifices) are to emit positive airflows can be determined. In someembodiments, the duration/length of time over which the selected subsetof diverting mechanisms is to perform the sorting action (which issometimes called a “firing duration”) that is determined by sortinglogic 504 is based at least in part on an attribute of the target item.For example, the duration/length of time that is selected by sortinglogic 504 is based at least in part on an approximate mass or anassociated geometry of the target item. For example, the larger the massor the larger the associated geometry of the target item, the greaterthe duration of time that would be selected by sorting logic 504 toperform the sorting action. One reason for sorting logic 504 to tailorthe selection of a particular subset of diverting mechanisms and alsothe duration over which the selected subset of diverting mechanisms isto perform a sorting action for each target item is to surgically aimthe sorting action on the target item to both successfully deflect thetarget item into a collection container (e.g., by using an appropriateforce to deflect the target item) and to minimize the probability ofinadvertently deflecting a nearby non-target item into a collectioncontainer (e.g., by focusing the force on the location of the targetitem). In some embodiments, the activating the selected subset ofdiverting mechanisms of the controllable array sorting device over thedetermined duration of time comprises sending one or more controlsignals to the controllable array sorting device to cause the sortingdevice to fire the selected subset of diverting mechanisms continuouslyfor the duration of time. In some embodiments, the activating theselected subset of diverting mechanisms of the controllable arraysorting device over the determined duration of time comprises sendingone or more control signals to the controllable array sorting device tocause the sorting device to fire the selected subset of divertingmechanisms intermittently for the duration of time. In some embodiments,the activating the selected subset of diverting mechanisms of thecontrollable array sorting device over the determined duration of timecomprises sending one or more control signals over the duration of timeto the controllable array sorting device to cause the sorting device tofire once or more per each received control signal. In some embodiments,the activating the selected subset of diverting mechanisms of thecontrollable array sorting device over the determined duration of timecomprises sending one or more control signals to the controllable arraysorting device to cause the sorting device to start firing the selectedsubset of diverting mechanisms in response to a start control signal andto stop firing in response to a stop control signal.

Sorting logic 504 is configured to determine whether to instruct to acontrollable array sorting device to either perform a sorting action ona target item or suppress (e.g., omit) the performance of the sortingaction on the target item based on the information (e.g., attributeinformation and location information) associated with that target itemand its neighboring non-target items that it receives from neuralprocessing logic 502. In some embodiments, sorting logic 504 isconfigured to determine, for each target item, whether the performanceof a sorting action on the target item (e.g., the removal of that targetitem from the stream of items) should be suppressed (e.g.,avoided/omitted) using a reconfigurable set of suppression criteria. Asdescribed above, it is possible that the physical deflection aimed atthe target item by the selected subset of diverting mechanisms couldinadvertently also deflect a non-target item into the collectioncontainer intended for the target item. Also, as described above, it isundesirable to allow neighboring non-target items to be inadvertentlydeposited into a collection container when a controllable array sortingdevice deflects a target item because doing so will lower the puritylevel associated with items collected at the collection container. Forexample, the set of suppression criteria describes the conditions forwhen a controllable array sorting device should not remove a target itemfrom the stream of items (to reduce the risk that the non-target itemalso inadvertently becomes deposited into a collection container). Insome embodiments, the set of suppression criteria may describe to firstdetermine for a target item, a set of neighboring non-target itemsrelative to the target item. In a first example, the set of neighboringnon-target items is determined as non-target items whose shortestdistances to the target item as measured between the respective polygonof the target item and the respective polygons of the non-target itemsare below a predetermined proximity threshold. In a second example, theset of neighboring non-target items is determined as non-target itemswhose shortest distances to the target item as measured between thecentroid of the target item and the respective centroids of thenon-target items are below a predetermined proximity threshold. In someembodiments, the set of suppression criteria may describe to thendetermine for a target item, a comparison (e.g., a ratio) between anattribute of the target item to an aggregate attribute associated withthe set of neighboring non-target items that had been determined for thetarget item. The comparison will be used by sorting logic 504 todetermine whether to instruct the controllable array sorting device toperform a sorting action on the target item or to suppress firing on thetarget item by not instructing the controllable array sorting device toperform a sorting action on the target item (to reduce the risk that thenon-target item becomes deposited into a collection container). Forexample, the approximate mass of the target item can be compared to theaggregated approximate masses of the set of neighboring non-target itemsand this ratio can be compared to a predetermined ratio threshold. Ifthe ratio is greater than the predetermined ratio threshold, sortinglogic 504 is configured to instruct the controllable array sortingdevice to perform the sorting action on the target item.

For the target items that sorting logic 504 determines should be firedon (e.g., removed from the stream of materials), in some embodiments,sorting logic 504 is configured to determine the manner in which thosetarget items are to be removed using a reconfigurable set of removalparameters. For example, the set of removal parameters describes, but isnot limited to, one or more of the following: which collectioncontainers to deposit target items of different material types and howmuch force/pressure to use to remove the target items from the streamand into corresponding collection containers. Sorting logic 504 isconfigured to send control signals to at least one correspondingcontrollable array sorting device to instruct that at least onecontrollable array sorting device which target items on which to performsorting actions and how to do so (e.g., using which subset of divertingmechanisms, how long to fire those diverting mechanisms for, and/orusing what degree of force to fire on the target items). In someembodiments, in response to detected events, tunable parametersassociated with the material recovery facility, including a set ofsuppression criteria and a set of removal parameters, can bereconfigured.

FIG. 5B is a diagram showing an example of a controllable array sortingdevice. In some embodiments, controllable array sorting device 100 ofFIG. 1 may be implemented using the example sorting device of FIG. 5B.In the example of FIG. 5B, the example controllable array sorting deviceincludes local controller 550 and array(s) of diverting mechanism(s)552. In various embodiments, local controller 550 may be implementedusing one or more processors or a microcontroller.

In some embodiments, array(s) of diverting mechanism(s) 552 are array(s)of air orifices that are connected to one or more sources of pressurizedair (not shown in FIG. 5B) via one or more manifolds (e.g., not shown inFIG. 5B). For example, an air orifice comprises a single air nozzle. Inanother example, an air orifice is an air knife that comprises a seriesof small apertures. The air orifices can be activated by localcontroller 550 to emit airflows that will deflect target items intorespective collection containers. For example, to activate airflows tobe shot from the air orifices, local controller 550 is configured tocontrol pressurized air to be supplied by one or more sources ofpressurized air. Where array(s) of diverting mechanism(s) 552 are arraysof air orifices, in some embodiments, different arrays of divertingmechanisms are pointing in different directions. The air orifices can beactivated by local controller 550 to shoot airflows continuously,intermittently, using a specified force of air, and/or for a specifiedduration of time. With respect to shooting airflows for a specifiedduration of time, the air orifices can be activated by local controller550 to shoot continuously for the specified duration of time, shootintermittently for the specified duration of time, shoot once or moreper each control signal over the specified duration of time, and startshooting in response to a start control signal and stop shooting inresponse to a stop control signal.

In some embodiments, array(s) of diverting mechanism(s) 552 are array(s)of mechanical paddles. For example, a mechanical paddle is a short bladethat is configured to swing/spring out in one direction and to retractback into its original position. The mechanical paddles can be activatedby local controller 550 to swing/spring out and physically make contactwith target items to deflect the target items into their respectivecollection containers. Where array(s) of diverting mechanism(s) 552 arearrays of air orifices, in some embodiments, different arrays ofdiverting mechanisms are pointing in different directions. Themechanical paddles can be activated by local controller 550 toswing/spring out and retract repeatedly, using a specified force ofswing, and/or for a specified duration of time. With respect to swingingmechanical paddles for a specified duration of time, the mechanicalpaddles can be activated by local controller 550 to swing continuouslyfor the specified duration of time, swing intermittently for thespecified duration of time, swing once or more per each control signalover the specified duration of time, and start swinging in response to astart control signal and stop swinging in response to a stop controlsignal.

Local controller 550 is configured to receive control signals from aninternal processor or an external/remote processor (e.g., associatedwith a cloud server or one or more sorting control devices) and tointerpret the received control signals to actuate array(s) of divertingmechanism(s) 552 to perform the instructed sorting actions on theappropriate target items. For example, local controller 550 candetermine when to actuate array(s) of diverting mechanism(s) 552 tofire, which subset of array(s) of diverting mechanism(s) 552 to fire,with what force (e.g., what pressure of airflows) array(s) of divertingmechanism(s) 552 should fire with, and/or for how long array(s) ofdiverting mechanism(s) 552 should continue to fire. In the specificexample in which array(s) of diverting mechanism(s) 552 comprisearray(s) of air orifices that are connected to pressured air sources,local controller 550 is configured to activate at least a subset of theair orifices of a specified array to shoot air at a particular time sothat the airflow can deflect a target item into a collection containerjust after that target item falls off a conveyor device.

FIG. 6A is a flow diagram showing an embodiment of a process for using acontrollable array sorting device. In some embodiments, process 600 isimplemented, at least in part, at a sorting control device such as thesorting control device that is included in FIG. 1 and sorting controldevice 500 of FIG. 5A. In some embodiments, process 600 is repeated asitems move along on a conveyor device and new sensed data is generatedbased on the current items that are located on the conveyor device.

At 602, sensed data associated with a set of items on a conveyor deviceis obtained. In some embodiments, sensed data comprises one or moreimage frames of the items on the conveyor device that are captured byone or more camera sensors that are pointed at the surface of theconveyor device.

At 604, a target item to be targeted by a controllable array sortingdevice is determined based at least in part on the sensed data, whereinthe controllable array sorting device comprises an array of divertingmechanisms. Attribute information and location information associatedwith the items identified in the sensed data are determined. Forexample, attribute information includes one or more of, but not limitedto, the following: a material type associated with each item, anapproximate mass associated with each item, an associated geometryassociated with each item, dimensions (e.g., height and width/area)associated with each item, a designated deposit (e.g., collectioncontainer) location associated with each item, and an orientationassociated with each item. For example, location information includesone or more coordinates at which each item was located in the imageframe(s). In some embodiments, an item is determined to be a “targetitem” based on the item's attribute and/or location information matchinga set of target item criteria.

At 606, at least a subset of the array of diverting mechanisms isselected based at least in part on an attribute of the target item. Atleast a subset of the array of diverting mechanisms of a controllablearray sorting device is selected to be used to perform a sorting actionon the target item. In various embodiments, the subset of the array ofdiverting mechanisms of a controllable array sorting device is selectedbased on an attribute (e.g., a geometry, an approximate mass) of thetarget item and a location of the target item on the conveyor device. Insome embodiments, where the controllable array sorting device includesmore than one array of diverting mechanisms and where the divertingmechanisms in different arrays are configured to deflect target items indifferent directions, the location of the selected number of divertingmechanisms is selected to be among a particular array that is configuredto deflect the target item into a direction in which a collectioncontainer corresponding to that target item is located. In someembodiments, a duration of time for which the selected at least subsetof diverting mechanisms is to perform the sorting action on the targetitem is also determined based at least in part on an attribute (e.g., ageometry, an approximate mass) of the target item.

At 608, an instruction to the controllable array sorting device is sentto cause the controllable array sorting device to use the selected atleast subset of the array of diverting mechanisms to perform a sortingaction on the target item. In some embodiments, performing a sortingaction on the target item is to deflect the target item into acorresponding collection container. In some embodiments, prior tosending an instruction (e.g., a control signal) to the controllablearray sorting device, it is determined whether the performance of thesorting action on the target item should be suppressed (e.g., avoided)in an effort to avoid inadvertently potentially deflecting non-targetitem(s) that are close in proximity to the target item into thecollection container of the target item along with the target item. Onlyif it is determined that performing the sorting action on the targetitem should not be suppressed is the instruction sent to thecontrollable array sorting device to cause the controllable arraysorting device to perform the sorting action on the target item usingthe selected at least subset of diverting mechanisms. In someembodiments, the instruction specifies the selected at least subset ofthe array of diverting mechanisms to use to perform the sorting action,the duration of time for which to perform the sorting action, a time orother temporal element for when to (start to) perform the sortingaction, and/or a degree of force (e.g., what pressure of airflows) touse in performing the sorting action. In some embodiments, theinstruction specifies to perform the sorting action while the targetitem is still on the conveyor belt of a conveyor device. In someembodiments, the instruction specifies to perform the sorting actionafter the target item has fallen off of a conveyor device.

FIG. 6B is a flow diagram showing an embodiment of a process forperforming a sorting action by a controllable array sorting device. Insome embodiments, process 650 is implemented, at least in part, at acontrollable array sorting device such as controllable array sortingdevice 100 that is included in FIG. 1 and the example controllable arraysorting device of FIG. 5B.

At 652, an instruction to perform a sorting action on a target item isprocessed, wherein the instruction specifies a selected at least subsetof the array of diverting mechanisms to use to perform the sortingaction. In various embodiments, the instruction is received from asorting control device. In some embodiments, the instruction isgenerated by a sorting control device using a process such as process600 of FIG. 6A. In some embodiments, the instruction is generated by aprocessor that is local to the controllable array sorting device. Insome embodiments, the instruction is received from a processor that isremote to the controllable array sorting device (e.g., a processorassociated with a cloud server or with another compute device within thesame facility). The instruction specifies which specific divertingmechanisms to use to perform the sorting action of deflecting the targetitem into a corresponding collection container. Where the controllablearray sorting device includes more than one array of divertingmechanisms, the instruction may further specify one or more array(s)that should perform the sorting action. In some embodiments, theinstruction specifies a duration of time for which to perform thesorting action. In some embodiments, the instruction specifies a degreeof force (e.g., the pressure of the airflows that are to be emitted fromdiverting mechanisms that are air orifices) to use to perform thesorting action. In some embodiments, the instruction specifies a time,start/stop control signal, or other temporal related element for whichthe controllable array sorting device is to perform the sorting action.

At 654, in response to the instruction, cause the selected at leastsubset of the array of diverting mechanisms to perform the sortingaction on the target item. The selected at least subset of the array ofdiverting mechanisms of the controllable array sorting device isactivated to perform the sorting action on the target item in accordancewith the zero or more other specified parameters of the instruction. Forexample, if the diverting mechanisms were air orifices, then, uponactivation, the selected at least subset of the air orifices would emitpressurized airflows to deflect the target item into its correspondingcollection container. In another example, if the diverting mechanismswere mechanical paddles, then, upon activation, the selected at leastsubset of the mechanical paddles would swing/spring out from theirinitial positions to make physical contact with the target item todeflect the target item into its corresponding collection container. Insome embodiments, the selected at least subset of the array of divertingmechanisms performs the sorting action while the target item is still onthe conveyor belt of a conveyor device. In some embodiments, theselected at least subset of the array of diverting mechanisms performsthe sorting action after the target item has fallen off of a conveyordevice.

FIG. 7 is a flow diagram showing an example of a process for determiningwhether to suppress performing corresponding sorting actions on a set oftarget items. In some embodiments, process 700 is implemented, at leastin part, at a sorting control device such as the sorting control devicethat is included in FIG. 1 and sorting control device 500 of FIG. 5A. Insome embodiments, process 600 of FIG. 6A is implemented at least inpart, using process 700.

At 702, sensed data associated with a set of items on a conveyor deviceis obtained. In some embodiments, sensed data comprises one or moreimage frames of the items on the conveyor device that are captured byone or more cameras or other sensors that are pointed at the surface ofthe conveyor device.

At 704, a set of target items and a set of non-target items areidentified from the sensed data. Attribute information and locationinformation associated with the items identified in the sensed data aredetermined. In some embodiments, an item is determined to be a “targetitem” if the item's attribute and/or location information matches a setof target item criteria and the item is determined to be a “non-targetitem” if the item's attribute and/or location information does not matcha set of target item criteria.

At 706, whether to instruct a controllable array sorting device toperform a corresponding sorting action on each of at least a subset ofthe set of target items is determined based at least in part on the setof non-target items. Whether a corresponding sorting action is to beperformed on each target item is determined based at least in part onthe non-target items that are located close enough to the target item onthe conveyor device to be considered “neighbors,” as will be furtherdescribed in FIG. 8, below.

FIG. 8 is a flow diagram showing an example of a process for determiningwhether to suppress performing a sorting action on a target item. Insome embodiments, process 800 is implemented, at least in part, at asorting control device such as the sorting control device that isincluded in FIG. 1 and sorting control device 500 of FIG. 5A. In someembodiments, step 706 of process 700 of FIG. 7 is implemented at leastin part, using process 800.

Process 800 shows an example of a process that determines whether theperformance of a sorting action on a target item should be suppressedbased on the particular set of neighboring non-target items that hasbeen determined specifically for that target item.

At 802, for a (next) target item, a set of neighboring non-target itemsis determined. For example, attribute information and locationinformation on the target item and the non-target items are identifiedby neural processing logic using sensed data associated with items on aconveyor device.

In a first example, a corresponding polygon is determined for the targetitem and for each non-target item, for example, using the locationinformation (e.g., centroid) of each item and the estimated geometry(e.g., length and width) of each item. Then, the respective shortestdistance between the polygon of the target item and the polygon of eachnon-target item is determined. Those non-target items whose respectiveshortest distances from their respective polygons to the polygon of thetarget item that are below a predetermined proximity threshold are thendetermined as the set of neighboring non-target items for that targetitem.

In a second example, the respective distance between the centroid of thetarget item and the centroid of each non-target item is determined.Then, those non-target items whose respective distances from theirrespective centroids to the centroid of the target item that are below apredetermined proximity threshold are determined as the set ofneighboring non-target items for that target item.

At 804, whether to suppress a sorting action on the target item isdetermined. In the event that suppression of a sorting action on thetarget item is determined, control is transferred to 806. Otherwise, inthe event that suppression of a sorting action on the target item is notdetermined, control is transferred to 810.

In some embodiments, a comparison between an attribute of the targetitem is compared to the aggregated attribute of the set of neighboringnon-target items that had been determined for that target item at step802. In a first example, the approximate mass of the target item iscompared to the aggregated/combined approximate mass of the set ofneighboring non-target items for that target item. Then, if the ratio isabove a predetermined threshold ratio, then the sorting action on thetarget item is determined to not be suppressed (e.g., because therelative mass of the target item to the collective mass of thenon-target items is significant, and that therefore, the target item isrelatively valuable and so the risk of inadvertently deflecting anon-target item into a collection container is acceptable). In a secondexample, the estimated volume of the target item is compared to theaggregated/combined estimated volume of the set of neighboringnon-target items for that target item. Then, if the ratio is above apredetermined threshold ratio, then the sorting action on the targetitem is determined to not be suppressed (e.g., because the relativevolume of the target item to the collective volume of the non-targetitems is significant, and that therefore, the target item is relativelyvaluable and so the risk of inadvertently deflecting a non-target iteminto a collection container is acceptable).

At 806, data associated with the target item is saved. If the sortingaction on the target item is determined to be suppressed, thatdetermination can be saved (e.g., and later analyzed to determinesubsequent suppression criteria).

At 808, instructing a controllable array sorting device to perform asorting action on the target item is omitted. If the sorting action onthe target item is determined to be suppressed, then the controllablearray sorting device is not instructed to fire on the target item andthe target item is allowed to land onto the next surface in the sortingline in which the controllable array sorting device is located.

At 810, a controllable array sorting device is instructed to perform asorting action on the target item. If the sorting action on the targetitem is not determined to be suppressed, then the controllable arraysorting device is instructed to fire on the target item after the targetitem falls off the conveyor device and deflect the target item into acorresponding collection container.

At 812, whether there is at least one more target item is determined. Inthe event that there is at least one more target item, control isreturned to 802 for the next target item. Otherwise, in the event thatthere are no more target items for which to determine whether acorresponding sorting action should be suppressed, process 800 ends.

FIG. 9 is a diagram showing an example of a set of neighboringnon-target items relative to a target item on a conveyor device. Asshown in the example of FIG. 9, at least items 904, 906, 908, and 910are currently located on the conveyor belt of conveyor device 900. Basedon sensed data with respect to the items (e.g., captured image frames ofthe items) on conveyor device 900, item 906 has been determined to be atarget item (e.g., by a sorting control device based on a set of targetitem criteria) while items 904, 908, and 910 have been determined to benon-target items (e.g., by a sorting control device based on a set oftarget item criteria). In determining whether a sorting action on targetitem 906 should be suppressed or not, in some embodiments, a set ofneighboring non-target items is first determined for target item 906.For example, a process such as process 800 of FIG. 8 can be used todetermine the neighboring non-target items for target item 906. In oneexample technique of determining the neighboring non-target items fortarget item 906, a polygon that approximates the shape of target item906 is determined and respective polygons that approximate therespective shapes of non-target items 904, 908, and 910 are alsodetermined. Then, the respective shortest distances between the polygonof target item 906 and the respective polygon of each of non-targetitems 904, 908, and 910 are determined. The respective shortestdistances between the polygon of target item 906 and the respectivepolygon of each non-target items 904, 908, and 910 are compared againsta predetermined proximity distance. Only the shortest distances betweenthe polygon of target item 906 and the respective polygons of non-targetitems 904 and 908 are determined to be less than the predeterminedproximity distance and so therefore, only non-target items 904 and 908(not non-target item 910) are determined to be neighboring non-targetitems relative to target item 906. As described by the example processof process 800 of FIG. 8, attribute information of target item 906 andaggregated attribute information associated with non-target items 904and 908 are compared to determine whether a sorting action should beperformed on target item 906 using a controllable array sorting device(that is not shown in FIG. 9 but would be located at the end of conveyordevice 900 from which items fall from conveyor device 900).

FIG. 10 is a flow diagram showing an example of a process for selectinga subset of diverting mechanisms of a controllable array sorting devicefor performing a sorting action on a target item. In some embodiments,process 1000 is implemented, at least in part, at a sorting controldevice such as the sorting control device that is included in FIG. 1 andsorting control device 500 of FIG. 5A. In some embodiments, step 606 ofFIG. 6A is implemented at least in part, using process 1000.

At 1002, a number of diverting mechanisms is selected based at least inpart on a first attribute of a target item. In some embodiments, thenumber of contiguous diverting mechanisms that is selected to performthe sorting action of deflecting the target item is selected based on anapproximate geometry (e.g., the width of the target item) and/or theapproximate mass of the target item. For example, a target item that hasa larger geometry and/or a larger mass will result in a larger number ofdiverting mechanisms being selected.

At 1004, a subset of diverting mechanisms in an array of a controllablearray sorting device corresponding to the selected number is selectedbased at least in part on location information associated with thetarget item and a corresponding collection container associated with thetarget item. Which of the diverting mechanisms associated with acontrollable array sorting device in the selected number of divertingmechanisms is to be selected is based on the location information (e.g.,the coordinate of the centroid) of the target item on the conveyordevice and also, the collection container corresponding to the targetitem. For example, the collection container corresponding to the targetitem may be determined by its material type and/or anothercharacteristic of the target item. If the controllable array sortingdevice includes more than one array of diverting mechanisms that areconfigured to deflect target items into different directions andtherefore, into different collection containers, then the subset ofdiverting mechanisms of the selected number is selected to be from thearray of the controllable array sorting device that is configured todeflect target items into the collection container that correspondsspecifically to the target item in question.

At 1006, the at least one of a duration of time and a force with whichto use the selected subset of the diverting mechanisms to perform asorting action on the target item is determined. In some embodiments,where diverting mechanisms can be activated to perform a sorting actionfor a certain duration of time (e.g., diverting mechanisms such as airorifices that deflect a target item via airflows that can becontinuously emitted for a given duration) and/or at a certain force(e.g., diverting mechanisms such as air orifices that deflect a targetitem via airflows for which the pressure of the airflows is adjustable),one or both of the duration and the force parameters can be selectedbased on another attribute of the target item. For example, theapproximate geometry (e.g., the length of the target item) and/or theapproximate mass of the target item can be used to determine one or bothof the duration and the force parameters that are to be used to performa sorting action on the target item.

At 1008, the controllable array sorting device is instructed to performthe sorting action on the target item using the selected subset of thediverting mechanisms using the determined at least one of the durationof time and the force.

FIG. 11 is a diagram showing an example relationship between the polygonof a target item and the determined firing width as well as the firingduration of a controllable array sorting device. In the example of FIG.11, two target items corresponding to bounding box 1102 and bounding box1104 are being transported by a conveyor device towards array ofdiverting mechanisms 1108. In the example of FIG. 11, the divertingmechanisms of array of diverting mechanisms 1108 are air orifices fromwhich pressurized air is emitted when the diverting mechanisms areactivated by the controllable array sorting device to perform a sortingaction of deflecting a target item into a collection container. In theexample of FIG. 11, the polygon of a target item is the bounding box ofthe target item. As such, the bounding boxes (bounding box 1102 andbounding box 1104) of two target items are shown in the example of FIG.11. For simplicity, this concept is represented by rectangular boxes inthe example of FIG. 11. However, the idea still of approximating theshape of an item still holds for more complex and/or precise contouringaround the item that may provide a tighter bound to the two-dimensionaland three-dimensional form of the item. For example, a minimum boundingbox is determined for each target item as the smallest bounding box thatencloses the entire target item. As mentioned above, an objectrecognition device that is located above a conveyor device that istransporting items to be sorted is configured to capture one or moreperspectives of the image frames of the items as they are moving on theconveyor device. A sorting control device that receives the capturedimage frames then applies machine learning to the image frames toidentify zero or more target items and zero or more non-target itemswithin the image frames. In some embodiments, for each identified targetitem within the image frames, the sorting control device determines acorresponding bounding box around the target item. The bounding box of atarget item will represent an approximate two-dimensional (e.g., lengthand width) or three-dimensional (e.g., length, width, and height)perspective of the target item. In some embodiments, the sorting controldevice will use the bounding box information of each target item, alongwith other information (e.g., other attribute information and thelocation information) of the target item to determine the “firing width”(i.e., the number and location of contiguous diverting mechanisms withinan array of diverting mechanisms of the controllable array sortingdevice) and the “firing duration” (i.e., the duration of time for whichthe selected diverting mechanisms within the firing width will performthe sorting action such as shooting airflows at the target item) thatshould be applied to the target item after it falls off the conveyordevice and is in proximity to the diverting mechanisms of thecontrollable array sorting device. In some embodiments, the firing widththat is selected for a target item increases as the size of the boundingbox of the target item increases. In some embodiments, the firingduration that is selected for a target item increases as the size of thebounding box of the target item increases. Referring to FIG. 11, becausebounding box 1104 is larger than bounding box 1102, both a wider firingwidth and a longer firing duration will be applied to firing on thetarget item associated with bounding box 1104 than the target itemassociated with bounding box 1102. The firing width and firing durationfor a target item are sent by the sorting control device to thecontrollable array sorting device via one or more control signals.

In some embodiments, other than the bounding box of a target item, thecontrollable array sorting device may use other factors to determine atleast the firing width and firing duration associated with a selectedsubset of diverting mechanisms to use on the target item. Examples ofthese other factors may include the determined material type of thetarget item, the approximate mass of the target item, the (e.g.,detected) speed of the conveyor device, and heuristics.

In various embodiments, performance of a sorting action on a target itemmay be suppressed depending on the neighboring non-target items relativeto the target item. In some embodiments, non-target items and theirrespective corresponding bounding boxes are also determined by thesorting control device from the image frame(s) that it has captured. Insome embodiments, the sorting control device determines the respectiveshortest distances between a bounding box of the target item and thebounding box of each one or more non-target items on the conveyordevice. The sorting control device is then configured to determine whichnon-target items are neighboring non-target items relative to the targetitem based at least in part on the respective shortest distance betweenthe bounding box of each non-target item and the bounding box of atarget item. For example, a non-target item for which the shortestdistance between its bounding box to that of the target item is lessthan a predetermined proximity threshold is considered to be aneighboring non-item relative to that target item. Then, attributeinformation of the target item is compared to attribute information ofthe same type corresponding to all neighboring non-target items relativeto the target item. For example, the approximate mass of the target itemis compared to the aggregated approximate masses of all neighboringnon-target items relative to the target item. The ratio of theapproximate mass of the target item to the aggregated approximate massesof all neighboring non-target items is then compared to a predeterminedratio threshold. For example, if the ratio is greater than thepredetermined ratio threshold, then a sorting action is determined to beexecuted on the target item. Otherwise, if the ratio is equal to or lessthan the predetermined ratio threshold, then a sorting action on thetarget item is determined to be suppressed. Referring to FIG. 11, giventhat the distance d between bounding box 1104 of the target item andbounding box 1106 of the non-target item is less than a predeterminedproximity threshold, the non-target item is determined to be aneighboring non-target item relative to the target item associated withbounding box 1104 in this example. Whether a sorting action on thetarget item associated with bounding box 1104 is to be suppressed maythen be determined based, for example, on a ratio between theapproximate mass of the target item relative to the at least approximatemass of the non-target item.

While FIG. 11 shows selecting the subset of diverting mechanisms of acontrollable array sorting device with respect to a target item based onthe determined bounding box of the target item, in some otherembodiments, the subset of diverting mechanisms of the controllablearray sorting device can be selected based on another form of geometryor other metric that is determined for the target item.

FIG. 12 is a diagram showing two example conveyor devices and acontrollable array sorting device that is located at the junctionbetween the two conveyor devices. Conveyor devices 1208 and 1210 areadjacent conveyor devices in a sorting line (e.g., in a materialrecovery facility, manufacturing process line, or otherwise). Items thatare transported across conveyor device 1208 are propelled towardconveyor device 1210 with trajectory 1202 due to the high speed at whichthe conveyor belt of conveyor device 1208 moves. In this example, thecontrollable array sorting device uses multiple rows/arrays of airorifices connected to pressurized air sources as diverting mechanisms toshoot target items in various directions out of trajectory 1202 intocorresponding collection mechanisms (not shown but could be targetconveyors or containers). For example, diverting mechanisms 1206 of thecontrollable array sorting device are configured to shoot target itemsdownwards into one or more collection mechanisms and divertingmechanisms 1204 of the controllable array sorting device are configuredto shoot target items upwards into one or more collection mechanisms.For example, different types of target items may be shot out oftrajectory 1202 into corresponding directions. That each of conveyordevices 1208 and 1210 is angled enables each to propel items fartherthan if each had been flat, thus providing more horizontal space at itsjunction with the next conveyor device for the controllable arraysorting device to deflect target items out of the propelled stream ofmaterials.

FIG. 13 is a diagram showing two example conveyor devices and twocontrollable array sorting devices that are located at the junctionbetween the two conveyor devices. In the example of FIG. 13, items aretransported along conveyor device 1308 a towards controllable arraysorting devices 1306 a and 1306 b. As the items are traveling acrossconveyor device 1308 a, object recognition device 1302 is configured tocapture one or more image frames of the items and a correspondingsorting control device (not shown) is configured to identify targetitems and also (e.g., neighboring) non-target items relative to eachtarget item. In some embodiments, the sorting control device isconfigured to determine whether the performance of a sorting action on atarget item should be suppressed based on attribute informationassociated with the target item and the attribute information of thattarget item's neighboring non-target items. For the target items onwhich the sorting control device determines that respective sortingactions should not be suppressed, the sorting control device isconfigured to send corresponding control signals to controllable arraysorting devices 1306 a and 1306 b to instruct the sorting devices tofire on the respective target items (e.g., using selected subsets ofdiverting mechanisms thereof). Controllable array sorting devices 1306 aand 1306 b are then configured to fire on the target items (for whichfire is not suppressed) as the target items are in the air in betweenconveyor device 1308 a and conveyor device 1308 b. As shown in theexample, controllable array sorting devices 1306 a and 1306 b are eachconfigured to shoot in two directions, where each direction isassociated with a particular material type or grouping of materialtypes. The fired-on target items are then deflected into theirrespective collection containers 1310 a, 1310 b, 1310 c, and 1310 d,where each collection container is configured to store target items of adifferent type of material, for example.

FIG. 14 is a diagram showing two example conveyor devices and twocontrollable array sorting devices that are located at the junctionbetween the two conveyor devices. The example of FIG. 14 is similar tothe example of FIG. 13, only that the two controllable array sortingdevices and collection containers are placed in different locationsrelative to each other at the junction/gap between the two conveyordevices.

FIG. 15 is a diagram showing an example of a detachable controllablearray sorting device. In some embodiments, a controllable array sortingdevice is detachable in that it can easily be inserted into or removedfrom a sorting line (e.g., in a material recovery facility,manufacturing process line, or otherwise). In the example of FIG. 15,detachable sorting device 1500 comprises manifolds 1506 that hold setsof valves that will be attached via pneumatic tubes (not shown in FIG.15) to array of diverting mechanisms 1508. In the example of FIG. 15,diverting mechanisms 1508 comprise air orifices that can be activated toemit airflows from pressurized air sources. Different sections of arrayof diverting mechanisms 1508 will shoot air at target items as they falloff a conveyor device, depending on where the target items were on theconveyor belt and the estimated dimensions (e.g., length and width) ofthe target items, to change the trajectory of the target items such thatthey fall onto a collection mechanism (e.g., a target conveyor or acollection container).

As shown in FIG. 15, sorting device 1500 is a specific example of acontrollable array sorting device that can be detachable from a sortingline because sorting device 1500 is not physically attached to the othermodular components (e.g., conveyor devices) of a sorting line. Instead,sorting device 1500 is configured to roll onto tracks 1504 that areattached to the floor of a material recovery facility. As such, sortingdevice 1500 can be easily taken out of a sorting line (e.g., formaintenance and repair) and then reinserted into the correct positionwithin the sorting line by simply being rolled back onto tracks 1504,which ensures that the sorting device will always be placed in the same,designated location. Handles 1502 located on the side of sorting device1500 make it easy for a user to pull and push the sorting device in andout of the sorting line.

Alternative to the detachable controllable array sorting device that canbe rolled onto tracks that is shown in FIG. 15, in some embodiments, thecontrollable array sorting device can be removable from a sorting linebut not attached to the ground. A specific example of a removablecontrollable array sorting device is one that can be attached, mounted,and/or fastened onto a conveyor device. For example, such a removablecontrollable array sorting device can be slid in and out of a designatedlocation along the conveyor device. That way, the removable controllablearray sorting device can still be unfastened from the conveyor devicefor repair or maintenance but can also be added back into the sortingline at a consistent position.

Any type of detachable controllable array sorting device such as theexamples described herein would enable the use of interchangeablesorting devices, which would help prevent any down time in operationwhen a controllable array sorting device is to be repaired ormaintained. Furthermore, in some embodiments, the controllable arraysorting device may include one or more components that enablemaintenance such as nozzle bar hinges.

FIG. 16 is a diagram showing an embodiment of an efficient materialrecovery facility with a single sorting line. Materials enter thesorting line at location 1602 and are serially processed by the seriesof nine conveyor devices (conveyor devices 1604 a, 1604 b, 1604 c, 1604d, 1604 e, 1604 f, 1604 g, 1604 h, and 1604 i) and interleavedcontrollable array sorting devices. Each of the controllable arraysorting devices may be implemented using controllable array sortingdevice 100 of FIG. 1. While not shown in the example sorting line ofFIG. 16, each of conveyor devices 1604 a, 1604 b, 1604 c, 1604 d, 1604e, 1604 f, 1604 g, 1604 h, and 1604 i can be angled at substantially thesame angle relative to the plane (e.g., floor) on which the conveyordevices are placed. The target items that are fired on by each of thecontrollable array sorting devices that follow a corresponding conveyordevice are deflected upwards, downwards, or laterally as they fall fromone conveyor device and onto a target conveyor that ultimatelytransports the target items away from the sorting line and into acorresponding collection container (such as collection containers 1612 aand 1612 b) that stores recovered material (e.g., of a particularmaterial type). The last controllable array sorting device in thesorting line is configured to select items of any material type that isselected by any of the preceding sorting devices and cause thoseselected items to be deposited onto recirculation conveyor 1610 so thatthey can be transported back to the front of the sorting line andrecirculate through the sorting line, starting from the first conveyordevice, conveyor device 1604 a. Those items that are not selected by thelast sorting device in the sorting line are deposited into a residuedisposal container (not shown).

A sorting device that includes a controllable array of divertingmechanisms is described herein. The array of diverting mechanisms can becontrolled to fire selective subsets of diverting mechanisms and forselective durations of time to deflect target items into collectionmechanisms. Various embodiments of a controllable array sorting devicecan be modularly added to sorting lines of a material recovery facilityto provide efficient and focused sorting of certain materials and typesof items.

Although the foregoing embodiments have been described in some detailfor purposes of clarity of understanding, the invention is not limitedto the details provided. There are many alternative ways of implementingthe invention. The disclosed embodiments are illustrative and notrestrictive.

What is claimed is:
 1. A controllable array sorting device, comprising:an array of diverting mechanisms; and a processor coupled to the arrayof diverting mechanisms, wherein the processor is configured to: processan instruction to perform a sorting action on a target item, wherein theinstruction specifies a selected at least subset of the array ofdiverting mechanisms to use to perform the sorting action; and inresponse to the instruction, cause the selected at least subset of thearray of diverting mechanisms to perform the sorting action on thetarget item.
 2. The controllable array sorting device of claim 1,wherein the array of diverting mechanisms comprises a first array ofdiverting mechanisms, and wherein the controllable array sorting devicecomprises a plurality of arrays of diverting mechanisms including thefirst array of diverting mechanisms, wherein each array of divertingmechanisms is configured to deflect items into a different direction. 3.The controllable array sorting device of claim 2, wherein theinstruction further specifies to use the first array of divertingmechanisms to perform the sorting action.
 4. The controllable arraysorting device of claim 1, wherein the instruction further specifies aduration of time for which to perform the sorting action on the targetitem and wherein the processor is further configured to cause theselected at least subset of the array of diverting mechanisms to performthe sorting action on the target item for the duration of time.
 5. Thecontrollable array sorting device of claim 1, wherein the instructionfurther specifies a first instruction to start performing the sortingaction and a second instruction to stop performing the sorting action onthe target item and wherein the processor is further configured to causethe selected at least subset of the array of diverting mechanisms toperform the sorting action on the target item in accordance with thefirst instruction and the second instruction.
 6. The controllable arraysorting device of claim 1, wherein the instruction further specifies afirst instruction and a second instruction, wherein each of the firstinstruction and the second instruction causes the selected at leastsubset of the array of diverting mechanisms to activate in associationwith performing the sorting action on the target item and wherein theprocessor is further configured to cause the selected at least subset ofthe array of diverting mechanisms to perform the sorting action on thetarget item in accordance with the first instruction and the secondinstruction.
 7. The controllable array sorting device of claim 1,wherein the instruction further specifies a degree of force with whichto perform the sorting action on the target item and wherein theprocessor is further configured to cause the selected at least subset ofthe array of diverting mechanisms to perform the sorting action on thetarget item using the degree of force.
 8. The controllable array sortingdevice of claim 1, wherein the array of diverting mechanisms comprisesan array of air orifices that are coupled to one or more pressurized airsources.
 9. The controllable array sorting device of claim 1, whereinthe array of diverting mechanisms comprises an array of mechanicalpaddles that are coupled to an actuation mechanism.
 10. The controllablearray sorting device of claim 1, wherein the instruction was determinedto be sent to the controllable array sorting device based at least inpart on a set of neighboring non-target items relative to the targetitem on a conveyor device.
 11. The controllable array sorting device ofclaim 10, wherein the set of neighboring non-target items relative tothe target item on the conveyor device was determined, including:wherein a respective distance between a polygon of the target item and arespective polygon of each of a set of non-target items was determined;and wherein a subset of the set of non-target items that is associatedwith the respective distances relative to the polygon of the target itemthat are less than a predetermined proximity threshold or a dynamicallydetermined and configured proximity threshold is determined as the setof neighboring non-target items relative to the target item.
 12. Thecontrollable array sorting device of claim 10, wherein the instructionwas determined to be sent to the controllable array sorting device basedat least in part on a comparison of attribute information associatedwith the target item and attribute information associated with the setof neighboring non-target items relative to the target item.
 13. Thecontrollable array sorting device of claim 1, wherein the at leastsubset of the array of diverting mechanisms comprises a number ofdiverting mechanisms that was selected based at least in part on anattribute associated with the target item.
 14. The controllable arraysorting device of claim 1, wherein the at least subset of the array ofdiverting mechanisms comprises a number of diverting mechanisms that wasselected based at least in part on location information associated withthe target item and a collection container corresponding to the targetitem.
 15. The controllable array sorting device of claim 1, wherein tocause the selected at least subset of the array of diverting mechanismsto perform the sorting action on the target item comprises to cause theselected at least subset of the array of diverting mechanisms to performthe sorting action on the target item after the target item fallsbetween conveyor devices.
 16. The controllable array sorting device ofclaim 1, wherein the controllable array sorting device is operable to beinserted and detached from a sorting line at a designated location. 17.The controllable array sorting device of claim 1, wherein theinstruction was determined based at least in part on using one or moreof neural networks and artificial intelligence on sensed data associatedwith the target item.
 18. A method, comprising: processing aninstruction to perform a sorting action on a target item, wherein theinstruction specifies a selected at least subset of an array ofdiverting mechanisms of a controllable array sorting device to use toperform the sorting action; and in response to the instruction, causingthe selected at least subset of the array of diverting mechanisms toperform the sorting action on the target item.
 19. The method of claim18, wherein the at least subset of the array of diverting mechanismscomprises a number of diverting mechanisms that was selected based atleast in part on an attribute associated with the target item.
 20. Themethod of claim 18, wherein the at least subset of the array ofdiverting mechanisms comprises a number of diverting mechanisms that wasselected based at least in part on location information associated withthe target item and a collection container corresponding to the targetitem.